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Executive Summary 

This research sought to determine which factors relate to higher and lower production 
of papers, citations and impacts (citations divided by papers), because this is one of the 
AAU Phase I Indicators (Appendix A) . The primaiy purpose of these analyses was to 
provide a better grasp of what factors appear to most influence citation productivity to 
serve as guiding principles as USF works toward becoming an AAU institution. 

Methods and Population 

The time period under consideration regarding citations was from 198 1 through 2005. 
In order to assess the relationship between various possible causal factors and citation 
productivity, comparisons among AAU and non- AAU institutions were conducted. AU 
institutions were dassifled as Research Extensive under the 2000 Carnegie system. 

Findings 

The following points emerge from these analyses: 

• It appears to reguire between 11 and 16 years for the average article to mature 
regarding dtations impact (Figure 1). 

• Over time, USF exhibits generally upward trends relative to all other institutions 
regarding dtation productivity (Table 4), however, the gap between USF and 
AAU institutions has not lessened during the time under consideration, except 
with regards to paper impact, where USF reached AAU levels by 1993 (Figure 9) . 

• The best simple predictors of dtation productivity are post dodorates, federal 
research expenditures, national academy members, total research expenditures 
and core revenues (Table 2). 

• The strongest influence on dtations is the percent of research conducted in 
biological and health sdences disdplines (Table 2). Panel B of Figure 2 shows 
that the broad disdpline areas of biological and health sdences, and other 
physical sdences and mathematics generate 80.5% of aU U.S. papers, and 89.7% 
of aU U.S. dtations. 

• Carnegie rankings exhibit a one- to- one relationship with funding (Figure 4), 
vdiich assodates with larger faculty numbers and greater research produdivity in 
the form of published papers and dtations (Figure 5). 

• AAU institutions average between two times and five times as many National 
Academy Members and Faculty Award winners as non- AAU Very High research 
institutions (Figure 7). 

• AAU institutions exhibit a 54% advantage over non- AAU Very High public 
research institutions for research expenditures per faculty member (Figure 8). 

• Both AAU and non- AAU Very High research institutions have about 30 % of their 
expenditures coming from grant- based funding. However, for USF this 
percentage was 42% (Panel B of Figure 6) . 

• Compared to Big East, SUS and Strategic Plan Peers, USF and other SUS 
institutions have lower tuition than other peer groups. USF has at least 10,000 
fewer dtations between 2001 and 2005 than the average research extensive 
institution of every peer group (Table 3) . 
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Discussion, Conclusions and Recommendations 

As one of the five Phase I AAU Indicators (Appendix A), citations is a prerequisite for 
attaining AAU membership. USF has generally been moving in the direction of AAU 
institutions for several years, however, as Birnbaum (2007) notes: "...'world- class' has 
increasingly come to be synonymous with 'Western.' That means science, research, and 
lots of money. . ." Regarding funding and revenues (money). Figure 4 depicts the gap 
between AAU and non- AAU research universities. USF is above the average non- AAU 
Very High public research institution, but below the average AAU institution on aU 
funding measures except tuition used in this study. 

Recently, USF has exhibited rapid growth in most of the areas that characterize typical 
public AAU institutions. However, regarding citations, although USF has exhibited 
growth, the gap between USF and the average AAU public institution has not decreased 
over the past 25 years (Figure 9) . The data in this study suggest that gap reduction 
requires more faculty, researchers and postdoctorates in the biological and health 
sciences or multidisciplinaiy sciences. The preceding groups are those who produce the 
most papers, which result in greater numbers of citations. 

Panel B of Figure 6 suggests that research at USF is more dependent on grant funding 
than the average institution from any other comparison group. This dependency may 
prove a detriment to future development, as more stable funding sources are usually 
preferable to uncertain sources such as grant funding. Overall, among 97 public 
research institutions, had the seventh highest percent coming from grants. However, UF 
was fourth, at 46 percent. Also, Florida was the seventh highest state overall for public 
research extensive institutions (33.5%), and would have been third (40.3%), except for 
FIU which was at 13 percent. Also, Hawaii at the top, was a single institution (Manoa). 

It appears that Florida's very high research institutions (UF, FSU, USF) have all been 
forc^ to turn to grant funds for growth and development. These data suggest that 
developing more dependable funding sources must be a top priority for USF during the 
coming years as we seek to attain AAU membership. 




Investigating the AAU Citations Criterion and the History of Papers, 

Citations and Impact at USF 
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Purpose 

This research investigates the AAU Phase I indicator Qtations (see Appendix A) . Several 
research questions guide the work, perhaps the most important of which is attempting 
to determine which factors relate to higher and lower production of papers, citations 
and impacts (citations divided by papers). Additionally, analyses of USFs current 
productivity and historical trends were investigated, as were those of AAU institutions. 
Big East research institutions. Strategic Plan Peers, SUS research institution 
competitors, and Carnegie High or Very High Research institutions. 

Research Questions Relating to Paper, Citation and Impact Productivity: 

Which factors relate more and less positively with citation productivity? 

How does USF relate to the following groups on recent productivity and discipline mix? 

• SUS Competitors 

• Big East Peers 

• Strategic Plan Peers 

• AAU pubic institutions 

• Non- AAU High or Very High Research institutions 

What can USFs historic grovdh trajectory tell us about future likelihoods? 

This is an extremely complex area within vdiich to conduct analyses because so many 
factors impact citations at the institutional level. The primary purpose of these analyses 
was to provide a better grasp of what factors appear to influence citation productivity to 
serve as guiding principles as USF works toward becoming an AAU institution. 

Background 

The AAU and the New Carnegie Classification Schema 
This research was primarily implemented in an attempt to determine ( 1) where USF 
stands relative to AAU institutions on citations, one of AAU's Phase I membership 
indicators (see Appendix A), and (2) what factors appear to relate more or less strongly 
with citation productivity. Further analyses compare USF with three additional relevant 
groups: Big East, SUS and Strategic Plan researdi institution peers. A secondary factor 
in these analyses relates to the fact that in December 2005, the Carnegie Foundation 
implemented a new, more varied and balanced method of classifying higher education 
institutions. Under the old method, all AAU institutions were classified as Research 
Extensive. Under the new method, all AAU institutions are classified in the Very High 
Research group. Therefore, this classification system was used as a basis for specific 
comparisons. The historic Carnegie Foundation Research Extensive classification was 
used as the basis for including non- AAU public institutions in analyses. 
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Methods 

Data Sources 

All data derive from one of the following three sources: 

1. The Thompson Science, Social Science and Humanities citations database 
(Thompson ISI), which provides extensive cumulative and individual year data 
from 1981 through 2005 at the institution and discipline within institution level. 

2. TheCenterTop American Research Universities (TARU) - TARU includes data 
on several important variables relative to this research for 639 institutions 
reporting some Federal Research funding during the past five years. 

3. IPEDS Peer Analysis System - Data were obtained for a variety of relevant 
variables from this source. 



Variables 

Outcome Variables 

Four outcome variables were used in analysis, each of which comes in multiple forms at 
the institution level: 

• Impact - Impact eguals citations divided by papers. This can be cumulative, 
annual and within ^sdpline. A secondary form of this is Impact of Qted, which 
reflects the total 

• Qtations - The number of citations: annual, cumulative and within discipline. 

• Papers - The number of published papers: annual, cumulative and within 
discipline. 

• Percent of Papers Qted - The percentage of all published papers receiving 
citations witfdn other publish^ papers. 

Categorical Variables 
Discipline 

TheTliompson ISI database classifies citations into 24 broad discipline areas (Table 1 ). 
These were reclassified into seven^ broader discipline areas for two reasons: ( 1) to 
increase population size within discipline, and (2) to make the data more amenable to 
analysis and to the available other sources of discipline-based information. IPEDS and 
TARU broad disciplines were reclassified into approximations of these seven discipline 
areas. The seven broad areas and included Thompson disciplines are: 

1. Biological & Health Sciences (Biology & Biochemistry, Clinical Medicine, 
Immunology, Microbiology, Molecular Biology & Genetics, Neurosdences & 
Behavior2, Pharmacology) 

2. Business 

3. Education 

4. Engineering & Computer Sciences 

5. Multi- Interdisdplinaiy 



1 Law was esxduded because USF lacks a law school. 

2 This was included here rather than Social Sciences because it is not strictly a social science. 
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6. Other Physical Sciences &Math (Agricultural Sciences, Chemistry, Ecology- 
environmental, Geosciences, Materials Science, Mathematics, Plant & Animal 
Science, Space Science) 

7. Social & Behavioral Sciences ( Social Sciences general. Psychology/ Psychiatry) 

The IPEDS disciplines were organized into parallel broad areas as follows: 

1. Biological and biomedical sciences. Health professions & related clinical sciences 

2. Business 

3. Education 

4. Engineering, Computer sciences. Engineering technologies. Military 
technologies, 

5. Multi- interdisciplinary studies 

6. Agricultural sciences. Mathematics & statistics. Physical sciences, 

7. Area- ethnic studies. Psychology, Protective services. Public administration. Social 
sciences, 

8. Other (Natural resources/ conservation. Communications &Technologies, 
En^ish, Foreign languages. Law, Liberal arts. Library science. Parks & 
recreation. Philosophy- religious studies. Visual & performing arts. History, 
Undesignated 

The TARU research disciplines included in five of the eight classifications are: 

1. Life sciences, 

2. NA 

3. NA 

4. Engineering & Computer sciences 

5. NA 

6. Physical sciences. Environmental sciences. Mathematics, 

7. Social sciences and Psychology 

8. Other sciences 

Institutional Group 

Institutional level data and group averages were developed for the following groups 
(Table 5 specifies institution names and affiliations). Most analyses included only 
institutions included within the 2000 Carnegie classification of Research Extensive: 

• AAU institutions - Includes all 34 public AAU institutions. 

• Big East institutions - Includes the 16 Big East institutions, although only 14 
are represented in the Thompson databases (Providence and Vrllanova have no 
records in Thompson's ISI). Ten Big East schools other than USE were 
classified as Research Extensive. 

• SUS institutions - Indudes UF, FSU and FlU in addition to USE. These were 
the four SUS Research Extensive institutions. 

• Strategic Plan Peers - Indudes the eight institutions identified as peers. 

• Other High or Very High Research Institutions - Includes 62 non- AAU 
institutions dassified by The Carnegie Foundation as either High or Very High 
Research. All were dassified as Research Extensive under the 2000 Carnegie 
system. 
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Independent Variables 

In addition to citations and papers from the Thompson I SI databases, the following 
variables were submitted to analysis (see Appendix B for descriptive statistics by 
variable): 



Predictor 


Source 


Total Student Enrollment fall 2005 


IPEDS 


Total Degrees Granted 2004-05 


IPEDS 


Core Revenues FY 2004 


IPEDS 


Total Grant Amount FY 2004 


IPEDS 


Grant Percent of Total Revenues FY 2004 


IPEDS 


Total Assets FY 2004 


IPEDS 


Instructional Expenditures FY 2004 


IPEDS 


Research Expenditures FY 2004 


IPEDS 


Total Faculty fall 2005 


IPEDS 


Faculty in Biological & Health Sdences^ 


IPEDS &TARU 


Faculty in Other Physical Sciences 


IPEDS &TARU 


Faculty in Social & Behavioral Sciences 


IPEDS &TARU 


Faculty in Education 


IPEDS &TARU 


Faculty in Engineering 


IPEDS &TARU 


Faculty in Business 


IPEDS &TARU 


Faculty in Multidisciplinary Disciplines 


IPEDS &TARU 


Faculty in Arts & Letters 


IPEDS &TARU 


Faculty in Fine Arts 


IPEDS &TARU 


Other Faculty 


IPEDS &TARU 


Pet of Total Research in Biological & Health Sciences 


TARU 


Pet of Total Research in Other Physical Sciences 


TARU 


Pet of Total Research in Social & Behavioral Sciences 


TARU 


Pet of Total Research in Engineering & Computer Science 


TARU 


Pet of Total Research in Other Fields 


TARU 


Pet of Federal Research in Biological & Health Sciences 


TARU 


Pet of Federal Research in Other Physical Sciences 


TARU 


Pet of Federal Research in Social & Behavioral Sciences 


TARU 


Pet of Federal Research in Engineering & Computer Science 


TARU 


Pet of Federal Research in Other Fields 


TARU 


National Academy Members 2003 2004 


TARU 


Faculty Awards 2003 2004 


TARU 


Doctorates Granted 2003 2004 


TARU 


Post Doctorates 2002 2003 


TARU 



Limitations 

Different data sources have been used for different aspects of similar measures, and 
particularly so, regarding funding and research expenditures. As a result, figures 
generated from one source are likely to differ from those coming from a different source. 
All figures have been verified, but tMs is the reason for the inconsistencies between 



3 Estimated from Percent of Research Expenditures by Institution and Total Eaculty 
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charts on funding and research dollars. Further, IPEDS and TARU sources may come 
from different years and also sometimes define apparently similar measures in different 
fashions, which cause apparent discrepancies. 

Some data are missing for some institutions on specific variables from different sources; 
therefore, the number of cases will differ depending on analyses. 

Because of the comparatively small populations (maximum 140), results for more 
complex analyses are open to guestion and were therefore generally avoided. This is an 
extremely complex area within which to conduct analyses because so many factors 
impact citations at the institutional level. The overall purpose of the analyses was to 
provide a better grasp of what factors appear to influence citation productivity to serve 
as guiding principles as USF works towaixi becoming an AAU institution. 

Results 

Population 

The population consisted of 140 institutions including aH public AAU members, and aH 
SUS and Big East institutions for which data were available. AU institutions, except for a 
few SUS and Big East schools, were Research Extensive under the 2000 Carnegie 
Classification system. For purposes of global analysis, only Research Extensive 
institutions are included, ^me institutions that were not Research Extensive are 
included where appropriate for specific analyses. 

Table 5 provides a list of institutions included in this study, along with their affiliations 
(e.g. Big East, AAU, etc.). Table 6 gives descriptive statistics for all variables included in 
the study. The skewness estimates shown in this table show that the use of 
nonparametric statistics was a better choice than parametric for aU tests of correlation 
magnitude. 

Citations 

Three factors are of interest regarding citations: 

1. Number of papers published in peer review) oumals 

2 . Number of times these papers are cited in other papers 

3. Impact eguals citations divided by papers either overall, or for a given year of 
papers. This measure controls for institutional size by providing the average 
number of citations each paper receives. The world wide average is 4.6 citations 
per published article. The greatest number of citations for any single article to 
2005 was over 293,000. One of every 58 papers receives 100 or more citations, 
one of every 270 papers receives 200 or more citations, and one of every 3,300 
receives 500 or more citations duringits influence period (Garfield, 2005). 

Because USF has less history than almost any AAU institution (other than U of 
California- Irvine), and because cumulative papers published and citations favor 
institutions with longer histories (greater productivity in earlier year^), most analyses in 
this study consider paper and citation productivity during the most recent five year 
period (2001 through 2005). 
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It is important to realize that two broad discipline areas, the biological and health 
sciences, and the other physical sciences and mathematics (includes engineering), 
generate both far more articles and far more citations per article than any other broad 
discipline area (see Figure 2 and Figure 3). 

Time Required for Maximum Citation Impact 

In an attempt to determine approximately how many years are required for articles to 
generate their maximum impact, a comparatively thorough analysis was conducted 
using Big East institutions due to the comparatively wide variety of institutions in the 
Big East. Data were not available for Providence College and VUlanova. We may assume 
that due to the diverse nature of the Big East, general trends that cross aU of the 
different types of Big East institutions are likely to hold for aU institutions. 

Figure 1 depicts 3-year rolling average^ impact curves for four Big East institutions of 
different types. Pittsburgh is an established AAU research university. USF is an up-and- 
coming research university. Syracuse is an established smaller research university, and 
St. J ohn's is primarily a literal arts university. The impact measure is the average 
number of citations for articles published in a given year. Of course, more recent articles 
(e.g. 2003 and 2005 - 2004 rolling average) havent had enough time to be cited very 
frequently. As the figure indicates, it takes roughly seven year^ for the average article to 
reach about two- thirds of its final impact. The highest impact point for USF and 
Pittsburgh is around 1990, suggesting that about 15 year^ is reguired for maximum 
impact ( 1990 to 2005). For 10 of the 14 Big East schools, the maximum article impact 
occurred between 1988 and 1994, with earlier articles not showing greater impact. For 
established institutions like Pittsburgh and Syracuse in the figure, article impact tends 
to be fairly flat over several year^ until it begins drepping as more recent articles havent 
had time to realize their full citation potential. 

One interesting effect is that for Syracuse, the maximum impact occurred from 1983 to 
1985 ( 1984 - rolling average), and that it dropped from there. This suggests that one, or 
a few articles from Syracuse written between 1983 and 1985 had a large number of 
citations and that vdien this article! s) ceased to be widely cited Syracuse's overall impact 
dropped. The same phenomena also occurred for Seton Hall, Connecticut and Notre 
Dame. Eleven of the Big East institutions show their maximum impact between 1989 
and 1994 which indicates that maximum impact occurs between roughly 11 and 16 year^ 
following publication. Most institutions show a slow growth from 198 1 ( 1982 here) to 
perhaps 1988, which may reflect either that an increase in the number of journals or the 
number of published articles occurred during that time, because more journals and 
articles mean more citations. Established institutions like Pittsburgh, Georgetown, 
Connecticut, etc. show comparatively fiat lines until the drop off begins at about 1995, 
when only 10 year^ of citations are available for an article. More recently published 
articles show fewer citations due to a lack of maturity. USF, as a grewing research 
university, showed steady growth in the number of citations from 198 1 threugh 1993. 



4 In order to stabilize the frequently erratic annual impact values, 3-year rolling averages were computed. The 
number for 1982 averages 1981 through 1983, and, for 2004, 2003 through 2005. 
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Pittsburgh and St. J ohns were chosen because they represent comparatively typical 
extreme cases regarding citations. Pittsburgh consistently shows about 10 more 
citations per article than USF, while St. J ohn's consistently shows about 10 fewer. After 
1988 Syracuse shows about five fewer citations per article than USF. These differences 
are consistent with overall cumulative 25 year impact data which is 24.0 for Pittsburgh, 
16.3 for USF, 14.8 for Syracuse and 8.6 for St. J ohn's. By 1999 (seven years of citations), 
USF was fourth in annual impact, behind Pittsburgh, Georgetown and Cincinnati and 
tied with Connecticut. By 2000, USF was tied with Qndnnati and had moved ahead of 
Connecticut. This upward trend of USF is reflected in the figure where one can see that 
as time moves forw^ to 2000 (the last legitimate year), USF's gap over Syracuse 
increases and gap beneath Pittsburgh decreases. 



Figure 1 

3-Year Rolling Average Impact Values 1981 through 2005 




' Syracuse — • — USF -a - Pittsburgh -A - St. John's 



Discipline Influence on Paper and Citations Productivity 

Table 1 lists the 24 broad discipline areas used by Thompson I SI to organize papers and 
citations. To more clearly show the discipline effects on papers and citations. Figure 2 
and Figure 3 depict respectively, impact and the percentage of papers and citations for 
seven very broad discipline areas that derive from the Thompson I SI 24 (see Methods 
for specifics). Three groupings are used in these figures: aU institutions, public AAU 
institutions and USF. AU institutions are used, rather than Very High and High 
Research Carnegie classifications because the issue here relates to generalizable 
phenomena. 



Table 1 

Thompson ISI 24 Broad Discipline Areas 



Agricultural Sciences 


Materials Science 


Biology & Biochemistry 


Mathematics 





12 



Chemistry 


Microbiology 


Clinical Medicine 


Molecular Biology & Genetics 


Computer Science 


Multidisciplinary 


Ecology/Environment 


Neurosciences & Behavior 


Economics & Business 


Pharmacology 


Education 


Physics 


Engineering 


Plant & Animal Science 


Geosciences 


Psychology/Psychiatry 


Immunology 


Social Sciences, general 


Law 


Space Science 



Figure 2 and Figure Sdisplay evidence indicating that the biological and health sciences 
and other physical sciences and mathematics broad discipline areas produce at least 
75% of all papers and citations. Panel B, the ^obal situation including all published 
papers from any U.S. source shows that 80.5% of papers and 89.7% of citations occur in 
those two broad discipline areas. A conspicuous paper and citation gap occurs between 
these and other disciplines. 

Generally, only two of the broad discipline areas show more citations than papers, those 
being the biological and health sciences and multidisciplinary. Regarding multi- 
disdplinaiy papers and citations, both figures suggest that USF more closely models 
public AAU institutions than all institutions with regards to that set of disciplines. This 
is further supported by Figure 3, which shows multidisciplinaiy to have the greatest 
number of citations per paper both for AAU institutions and USF, but not for all 
institutions, where the biological and health sciences are greatest. This may partially 
occur because of multidisciplinaiy papers involving the social and behavioral sciences, 
engineering and physical sciences and biological and health sciences, all of which are 
comparatively high on citations. 

One phenomenon that Figure 2 and Figure 3 suggest is that USF produces 
comparatively fewer citations in the other physical sciences and mathematics very broad 
discipline areas than either the average AAU or average research extensive institution 
does. 
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Figure 2 



AAU - Percent of All Papers and Citations by Very Broad Disciplines - 1981 to 2005 




I □ % of Citations ■ % of PapeTs] 



All - Percent of All Papers and Citations by Very Broad Disciplines - Cumulative 
1981 to 2005 




I □ % of Citations ■ % of Paper^ 



USF - Percent of All Papers and Citations by Very Broad Disciplines - 1981 to 2005 




I □ % of Citations ■ % of Paper^ 
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Figure 3 



AAU - Average Number of Citations (Impact) Per Published Paper 
by Very Broad Discipline - 1981 to 2005 
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What Institutional Characteristics Best Predict Paper and Citation Productivity? 

Because Appendix B shows that the several of the value distributions for citations and 
predictor measures exhibit comparatively extreme distributional characteristics, robust 
rank statistical comparisons were applied rather than traditional Ordinary Least Square 
(OLS) analyses. In order to evaluate simple relationships (correlations) between 
predictor measures and citation productivity output variables (papers, citations and 
impact), Spearman Rranks correlations were computed for the complete population of 
140 institutions for which aU data were available. Table 2 exhibits the results of these 
correlations. Before discussing these correlations, it becomes necessary to discuss 
multiplicity. Because so many analyses were conducted, rather than considering the 
statistical significance of relationships, the following discussion wiU be concerned only 
with the magnitude and direction of obtained relationships. ^ 

Table 2 ranks predictors in descending order by the magnitude of relationship with the 
total number of citations. The highest correlations/ relationships with each outcome are 
highlighted in yellow. The sin^e best predictor of citations is the number of post 
doctorates, followed fairly closely by the amount of federal research expenditures, the 
number of national academy members, total research expenditures and core revenue 
amount. None of these top relationships differ significantly among themselves because 
aU are strong. All of the predictors relate at a 0.60 or higher rate to the number of 
papers and citations down to total faculty numbers. Relationships with citation impact, 
however, drop off far more quickly, reducing below Rranks 0.60 after total grant 
revenues. Total enrollment and total degrees are the only variables exhibiting zero or 
negative impacts, and that only occurs for impact and percent of papers dted. 

The bottom section of the table shows relationships between the percentage of research 
expenditures in TARU broad discipline areas and the citations outcome variables. Other 
than research in the biological and health sciences, all of these relationships are negative 
or near zero. This indicates that the strongest influence on citations is research 
conducted in the biological and health sciences disciplines. 



Spearman Ranks Correlations of Ci 



Table 2 

ation Productivity Predictors with Five Outcomes 







5-year 


Impact 


Num 


Der of 


Percent 




N 


Impact 


of Cited 


Papers 


Citations 


Cited 


Post Doctorates Fall 2003 


132 


0.76 


0.74 


0.88 


0.91 


0.79 


Federal Research FY 2004 


132 


0.71 


0.69 


0.90 


0.90 


0.71 


Academy Members 2005 


132 


0.75 


0.74 


0.86 


0.88 


0.76 


Total Research FY 2004 


132 


0.57 


0.55 


0.92 


0.87 


0.59 


Core Revenues FY 2004 


140 


0.62 


0.61 


0.89 


0.87 


0.59 


Expenditures- Research FY 04 


134 


0.64 


0.63 


0.88 


0.87 


0.63 


Faculty Awards FY 2004-05 


132 


0.70 


0.69 


0.85 


0.86 


0.69 


Total Grants FY 2004 


140 


0.60 


0.59 


0.88 


0.86 


0.59 



^ Given a population of 140, all rranks correlations of 0.30 or higher are significant at the p < .0001 level for single 
comparisons; however, due to the multiplicity involved, significance is comparatively meaningless. 
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5-year 


Impact 


Num 


Der of 


Percent 




N 


Impact 


of Cited 


Papers 


Citations 


Cited 


Expenditures-Instruction 


134 


0.55 


0.55 


0.82 


0.80 


0.53 


Total Doctorates AY 2005 


132 


0.41 


0.41 


0.84 


0.76 


0.41 


Assets FY 2004 


90 


0.52 


0.50 


0.73 


0.73 


0.53 


Total Faculty Fall 2005 


140 


0.31 


0.31 


0.67 


0.61 


0.31 


Faculty Bio- Health Sciences 


140 


0.39 


0.37 


0.59 


0.57 


0.43 


Faculty Engineering 


140 


0.17 


0.17 


0.57 


0.48 




Total Degrees AY 2005 


140 


-0.04 


-0.04 


0.48 


0.35 




Grant % of Revenues FY 2004 


140 


0.26 


0.25 


0.33 


0.32 




Total Enrollment Fall 2005 


139 


-0.18 


-0.18 


0.33 


0.19 




Percent of Research Expenditures by TARU Broad 


Discipline Area, FY 2004 


Biological & Health Sciences 


132 


0.39 


0.36 




0.43 


0.44 


Other Physical Sciences 


132 


-0.20 


-0.20 




-0.24 


-0.22 


Engineering & Computer Sd 


132 


-0.30 


-0.28 




-0.19 


-0.36 


Social & Behavioral Sciences 


130 


-0.21 


-0.20 


-0.16 






Other Sciences 


130 


-0.23 


-0.23 


-0.11 







Analyses and Comparisons with AAU Institutions 

Figure 4 through Figure 8 provide some direct comparisons between public AAU 
institutions, USF, and other public Carnegie Very High and High Doctoral/ Research 
Institutions. The population sizes are AAU, 34, Very High Research excluding USF, 27, 
and High Research 35. All institutions included in these populations were classified as 
Research Extensive under the 2000 Carnegie Classification System. All data are for the 
most recently available years. 

Figure 4 shows that the funding and revenue available to AAU institutions is greater 
than at non- AAU institutions, although not as great a difference occurs for grant funds 
as for core revenues. This figure shows that the Carnegie rankings exhibit a one-to-one 
relationship with funding. Figure 5 depicts how these additional revenues translate into 
larger faculty numbers which in turn associates with greater research productivity in the 
form of published papers and citations. 

Figure 6 depicts the fact that non- AAU institutions are far closer to AAU institutions in 
the impact of citations (panel A) and in the percentage of papers that are cited (panel B), 
than in total citations. Impact reflects the average number of citations per paper. Panel 
B displays the fact that, of the papers that are cited (between 64% and 70% for these 
institutions) AAU cited source papers are cited perhaps 25% more frequently than are 
those from High Research institutions, and about the same as non- AAU Very High 
Research institutions. USF's impact and percent cited are both relatively close to the 
average AAU institution (see Table 2 for the relevant dates) . 



Figure 7 shows that AAU institutions have a large advantage regarding the number of 
National Academy Members and Faculty Award winners over non- AAU members. Panel 
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B tells that a similar advantage ocmrs regarding the number of doctorates granted and 
post-doctoral appointments at these institutions. 

Panel A of Figure 8 suggests that institutional size may be an important factor in the 
funding and faculty numbers results. Therefore, Panels B and C standardize such values 
relative to the size of the institution's student body and faculty numbers. The AAU 
institutions exhibit less of an advantage relative to each student or each faculty member 
regarding core revenues and grant dollars expended in the most recent year. Panel C 
indicates that the same is true regarding instructional expenditures per FTE student. 
However, regarding research expenditures per faculty member, the AAU institutions 
retain a 54% advantage over non- AAU public Very High research institutions, and a 
greater advantage over High Research universities. 

Generally, USF's values consistently fall close to the non- AAU Very High Research 
institutions. Regarding grant funding, USF is more dependent on this source of income 
than are AAU, Very High or High Research institutions and this may prove a detriment 
to future development. Overall, among 97 public research institutions, USF had the 
seventh highest percent of funds coming from grants. However, UF was fourth, at 46 
percent while FSU had 35 percent. Overall, Florida was the second highest state for 
public Very High institutions (41%) behind only Hawaii with a single institution, Manoa, 
at 47 percent. It appears that Florida's very hi^ research institutions (UF, FSU, USF) 
have either chosen to or been forced to turn to grant funds for growth and development. 
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Figure 4 

Funding and Revenues 2004 
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Figure 5 

Number of Faculty, Tenure Track and Total - Fall 2005 
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Figure 6 

Public Institutions 5-Year (01-05) Citations Impact 
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Figure 7 



National Academy Members and Faculty Awards 
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Figure 8 

FTE Enrollment and Degrees Granted - AY 2005 
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Analyses and Comparisons with Big East, SUS and Strategic Plan Peers 
In tlie following comparisons, only Research Extensive institutions are compared.® The 
Big East Research Extensive institutions are: the University of Connecticut Georgetown 
University, the University of Notre Dame, the University of LouisviUe, The University of 
New Hampshire, Rutgers University, Syracuse University, The University of Pittsburgh, 
West Virginia University and Marguette University. The three SUS institutions are The 
University of Florida, Florida State University and Florida International University. 

Note that some of these institutions have lower values than others and puU down the 
mean values. For example, both Florida International University and Florida State 
University have far lower values than the University of Florida on aU of the measures 
included in Table 3. USF's values are generally comparable with these sets of peers and 
generally below the Strategic Plan (SP) peers on the measures shown, with a few 
exceptions: 

1. Tuition - Both USF and other SUS institutions have lower tuition than the other 
peers. Note that some of the Big East institutions are private. 

2. Citations- USF has at least 10,000 fewer citations during the five year period 
from 2001 through 2005 than any of the other group means. 

3. Grants as a percentage of funds - For USF, grants represent between a 35% 

(SUS) and 83% (Big East) greater percentage of total revenues than at any of the 
other peer group's average institution (note, 42%/ 23% = 182.6%) . 

4. National Academy Members and Faculty Awards - USF is below aU other group 
means on these Phase I AAU Membership Indicators. UsuaUy, USF is closer than 
the most recent year with regards to Faculty Awards. 

Table 3 



Comparison of USF wi 


th Big East, Strategic Plan and SUS Peers 




USF 


Big East 


SP Peer 


SUS Peers 


N of Institutions 


1 


10 


8 


3 


Citations 5-yr (2001-2005) 


32,389 


42,692 


56,284 


43,243 


Papers 5-yr (2001-2005) 


5,035 


6,058 


8,292 


8,238 


Impact 5-yr (2001-2005) 


6.4 


6.2 


6.5 


5.0 


Impact of Cited (2001-2005) 


9.7 


9.1 


9.2 


7.8 


PercentCited (2001-2005) 


66% 


68% 


69% 


64% 


Core Revenues FY04 


42% 


23% 


29% 


31% 


Grants as a Percentage of Funds FY04 


$842,461,077 


$730,854,855 


$905,416,151 


$979,015,883 


Totai Grant Amount FY04 


$353,833,652 


$176,782,626 


$259,139,710 


$365,908,450 


Instructionai Expenditures FY04 


$256,729,160 


$191,267,219 


$270,359,444 


$263,573,865 


Research Expenditures FY04 


$39,218,817 


$30,657,133 


$38,255,734 


$47,452,094 


Federai Research FY04 


$126,906,000 


$94,712,100 


$160,690,875 


$124,259,333 


Totai Research FY04 


$240,021,000 


$141,387,000 


$272,641,625 


$226,274,333 


Tuition Si Fees 0607 


$3,416 


$17,397 


$7,153 


$3,214 


Tenured/Ten Track F acuity 2005 


1,399 


1,000 


1,316 


1,658 


Totai Facuity 2005 


2,697 


2,010 


2,530 


2,779 


Nationai Academy Members 


2.0 


7.9 


14.6 


9.3 



6 This comparison includes private Big East institutions to increase the numbers in each cell. 
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USF 


Big East 


SP Peer 


SUS Peers 


N of Institutions 


1 


10 


8 


3 


Facuity Awards 


5.0 


9.7 


14.4 


9.7 


Post Doctorates 03 


127 


119 


254 


219 


Totai EnroiimentFaiiOS 


42,660 


20,088 


25,943 


41,914 


FTE Enroiiment0506 


27,786 


16,827 


20,823 


32,584 


Totai Degrees 0506 


8,336 


4,690 


5,742 


9,677 


Doctorates Granted 0506 


194 


174 


281 


353 



Changes Over Time at USF 

Table 4 provides the number of papers published by broad discipline area in 198 1-85 
and 2001-05. The overall growth of papers published at USF was 188% from 1,964 in 
the earlier years, to 5,649 in the most recent 5-year period. This compares with an 
increase of 76% for all U.S. paper sources. Below the total row, broad disciplines are 
divided into two groups: ( 1) those at USF publishing more than 100 papers between 
2001 and 2005, and (2) those publishing fewer than 100 papers duringthat time. 
Within these groups, disciplines are sorted by growth from greatest to least. Looking at 
USF's growth, most disciplines show at least a doubling of published papers ( 100% 
grovdh). This is a greater growth than average, because, among aU U.S. sources, only six 
of the 24 discipline areas show 100% or greater growth, and only computer science 
shows more than a 200% growth. At USF, 15 disciplines showed 100% or more, and 11 
showed a 200% or greater increase. Only in mathematics and space science does USF 
show less growth than the overall U.S. rate. 

For USF, the growth percentage relates to the number published during the early 
period, as Clinical Medicine, which published almost three percent of aU papers from 
198 1 to 1985, showing average growth, although they increased their publication by 
almost 1,300 papers. Physics and Materials Science, which respectively published 23 
and 3 papers from 198 1 to 85, showed the greatest growth, respectively 787% and 
2000% with increases respectively of 18 1 and 60 papers. Biology and Biochemistry 
showed about the lowest growth (62%), although they increased their publications by 
147 papers. 



Table 4 

Changes in the Number of USF Published Papers by Discipline from 81-85 to 01-05 







Total Papers 


Growth 




25yr Totals 


1981-1985 


2001-2005 


USF 


All U.S. 


All Disciplines 


19,745 


1,964 


5,649 


188% 


76% 


USF Disciplines Publishing More Than 100 Papers from 2001 to 2005 


Physics 


466 


23 


204 


787% 


113% 


Molecular Biology & Genetics 


443 


22 


139 


532% 


130% 


Neurosciences & Behavior 


857 


52 


303 


483% 


99% 


Engineering 


863 


51 


274 


437% 


124% 


Geosciences 


749 


64 


255 


298% 


97% 


Immunology 


663 


40 


141 


253% 


74% 


Social Sciences, general 


1,330 


126 


429 


240% 


68% 
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Total Papers 


Growth 




25yr Totals 


1981-1985 


2001-2005 


USF 


All U.S. 


Economics & Business 


340 


36 


110 


206% 


63% 


Ciinicai Medicine 


6,606 


594 


1,794 


202% 


81% 


Ecology/Environment 


455 


52 


150 


188% 


133% 


Psychoiogy/Psychiatry 


1,568 


167 


455 


172% 


42% 


Chemistry 


868 


113 


245 


117% 


76% 


Biology & Biochemistry 


1,551 


236 


383 


62% 


36% 


Plant & Animal Science 


994 


162 


259 


60% 


33% 


USF Disciplines Publishing Fewer Than 100 Papers from 2001 to 2005 


Materials Science 


142 


3 


63 


2000% 


137% 


Computer Science 


197 


14 


68 


386% 


213% 


Multidisciplinary 


240 


25 


59 


136% 


6% 


Education 


183 


22 


48 


118% 


8% 


Pharmacology 


331 


41 


74 


80% 


28% 


Agricultural Sciences 


94 


13 


23 


77% 


19% 


Microbiology 


284 


44 


74 


68% 


58% 


Mathematics 


473 


56 


90 


61% 


79% 


Law 


37 


6 


9 


50% 


15% 


Space Science 


11 


2 


0 


-100% 


94% 



USF Relative to the Average AAU Institution on Papers and Citations 
Figure 9 depicts that, relative to annual papers published and citations, USF roughly 
parallels the growth exhibited by the "Average" AAU institution between 1981 and 2005. 
However, USF's parallel course is significantly lower than that of the average AAU 
institution. Figure 1 and the discussion regarding the time reguired for maximum 
impact explain vdiy citations and impact exhibit downward trends after about 1993. 



Regarding impact, by 1993, USF reached that of the "average" AAU institution, and 
although there as a dip relative to the AAU in 1994 and 1995, by 1997, USF was again 
egual to the "average" AAU institution for this metric. 
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Figure 9 

Mean and Median AAU - Annual Papers Published 




Mean and Median - AAU Annual Number of Citations 




^ AAU Mean o AAU Median USF| 



Mean and Median - AAU Annual Impact 
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Summary and Discussion 

As one of the five Phase I AAU Indicators (Appendix A), citations is a prerequisite for 
attaining AAU membership. USF has generally been moving in the direction of AAU 
institutions for several years, however, as Birnbaum (2007) notes: . .'world- class' has 
increasingly come to be synonymous with 'Western.' That means science, research, and 
lots of money. . ." Regarding funding and revenues (money). Figure 4 depicts the gap 
between AAU and non- AAU research universities. USF is almost equidistance between 
the mean public AAU and non- AAU institutions regarding funding and revenues. 

In recent history, USF has exhibited rapid growth in most of the areas that characterize 
typical public AAU institutions. Unfortunately, regarding citations, although USF has 
ej^bited growth, the gap between USF and the average AAU public institution has not 
decreased over the past 25 years (Figure 9) . The data in this study suggest that gap 
reduction requires more faculty, researchers and postdoctorates in the biological and 
health sciences or multidisciplinaiy sciences. The preceding groups are those who 
produce the most papers, which result in greater numbers of citations. 

Panel B of Figure 6 suggests that research at USF is more dependent on grant funding 
than either AAU or other Very High or High Research institutions. This dependency 
may prove a detriment to future development, as more stable funding sources are 
usually preferable to variable funding sources such as grant funding 
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Appendix A 

AAU Membership Indicators and Definitions 



Phase I Indicators 

1) Competitively funded federal research support: These data are collected by the 
National Science Foundation. The Membership Committee has been using obligations, 
vdiich are the only measures that break down federal support by agency. The committee 
has recently switched to using NSF research expenditure data, which are more accurate, 
with a correction factor to subtract the estimated proportion of university expenditures 
drawn from USDA. Most USDA funding is not allocated competitively, and USDA 
support accordin^y is included as a Phase II indicator. 

2) Membership in the National Academies (NAS, NAE, and lOM): The National 
Academies' membership database maintains the current institutional affiliation of its 
members. 

3) National Research Coundl faculty guality ratings: These ratings are drawn from the 
decennial national assessment of research- doctorate programs conducted by the NRC. 
Though the data become dated between surveys, the committee believes that they 
continue to provide a valuable peer- assessment of faculty guality. The last NRC report 
was published in 1995 based on 1993 data; preparation for the next NRC assessment is 
currently underway. 

4) Faculty arts and humanities awards, fellowships, and memberships: For its last 
research doctorate assessment, NRC compiled a list of awards, fellowships, and 
memberships signifying faculty achievement primarily in arts and humanities fields. 

The Membership Committee has expanded this list and will use it as an additional 
assessment of the distinction of an institution's faculty, focusing on the arts and 
humanities faculty (Attachment 1). Additional appropriate awards, fellowships, and 
memberships wUl be added to this list as they are identified. 

5) Qtations: The U.S. University Science Indicators citations database provides an 
annually updated measure of both research volume and guality and will provide a 
valuable complement to the first four indicators listed above. 

Phase II Indicators 

1) USDA, state, and industrial research funding: Though these three sources of 
academic research support fund important, high- guality research, they will be treated as 
phase II indicators since they are generally not allocated through competitive, merit- 
review processes. Competitively funded USDA research programs that can be separately 
identified in reported data will be included in phase I data. 

2) Doctoral education: The committee will use number of Ph.D.s granted annually as 
well as tabulate the distribution of Ph.D.s across broad disdplinaiy categories (e.g., 
engineering but not aerospace engineering), using Department of Education IPEDS 
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(Integrated Postsecondaiy Education Data System) data. These data will be treated as 
phase II indicators to de-emphasize the quantitative dimensions of Ph.D. programs and 
avoid sending an unintended signal to institutions to increase Ph.D. output at a time 
vdien many institutions are or are considering scaling back their Ph.D. programs. 

3) Number of postdoctoral appointees: The committee will use NSF- compiled data 
from institutions on postdoctoral appointees, most of who are in the health sciences, 
physical sciences, and engineering. Postdoctoral education is an increasin^y important 
component of university research and education activities that the committee believes 
should be tracked in AAU membership indicators. However, because postdoctoral 
activity is highly correlated with university research and because self-reported 
postdoctoral data are less uniform than data on federally funded research, postdoctoral 
appointees will be treated as a phase II indicator. 

4) Undergraduate education: The committee will assess the institution's undergraduate 
programs to determine that the institution is meeting its commitment to undergraduate 
education. Recognizing that differing institutional missions among research universities 
dictate different ways of providing undergraduate education, the committee will be 
flexible in this assessment. A number of measures have been suggested, including some 
that focus on input and others that look primarily at output variables. These are at this 
time imperfect, but may provide some guidance to the committee in making its 
judgments on this topic. 
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Appendix B 



Table 5 

Institutions Included in the Study 



Public 


Private 


AAU Institutions 


Indiana U-Bloomington 




Brandeis U 




Iowa State U 




Brown U 




Michigan State U 




Carnegie Mellon U 




Ohio State U 




Case Western Reserve U 




Pennsylvania State U 




Columbia U New York 




Purdue U 




Cornell U 




Rutgers U-New Brunswick 


Big East, 
SP Peer 


Duke U 




SUNY-Buffalo 


SP Peer 


Georgetown U 


Big East 


SUNY-Stony Brook 


SP Peer 


Harvard U 




Texas A & M U 




Johns Hopkins U 




The U of Texas-Austin 




MIT 




U of Arizona 




Marquette U 


Big East 


U of California-Berkeley 




New York U 




U of California-Davis 




Northwestern U 




U of California-Irvine 


SP Peer 


Princeton U 




U of California-Los Angeles 




Rice U 




U of California-San Diego 




Stanford U 




U of California-Santa Barbara 




Syracuse U 


Big East 


U of Colorado-Boulder 




Tulane U of Louisiana 




U of Florida 


SUS 


U of Chicago 




U of Illinois-Urbana-Champaign 




U of Pennsylvania 




U of Iowa 




U of Rochester 




U of Kansas 




U of Southern California 




U of Maryland-College Park 




Vanderbilt U 




U of Michigan-Ann Arbor 




Yale U 




U of Minnesota-Twin Cities 








U of Missouri-Columbia 








U of Nebraska-Lincoln 








U of North Carolina-Chapel Hill 








U of Oregon 








U of Pittsburgh 


Big East 






U of Virginia 








U of Washington-Seattle 








U of Wisconsin-Madison 








Non AAU Institutions 


Arizona State U-Tempe 




Boston U 




Clemson U 




California Institute of Technology 




Colorado State U 




Dartmouth College 




Florida Agricultural & Mechanical U 


SUS 


DePaul U 


Big East 
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Public 


Private 


Florida Atlantic U 


SUS 


Emory U 




Florida International U 


SUS 


Providence College 


Big East 


Florida State U 


SUS 


Rensselaer Polytechnic 




Georgia Institute of Technology 




Saint Johns U 


Big East 


George Mason U 




Seton Hall U 


Big East 


Georgia State U 




Tufts U 




Kansas State U 




U of Miami 




Louisiana State U 




U of Notre Dame 


Big East 


Mississippi State U 




Villanova U 


Big East 


Montana State U-Bozeman 




Washington U in St Louis 




New Mexico State U 




Yeshiva U 




North Carolina State U-Raleigh 


SP Peer 






Northern Illinois U 








Oklahoma State U 








Old Dominion U 








Oregon State U 








SUNY-Albany 








San Diego State U 








Southern Illinois U Carbondale 








Temple U 








Texas Tech U 








The U of Alabama 








The U of Tennessee 








The U of Texas-Arlington 








U of Alabama-Birmingham 


SP Peer 






U of Arkansas 








U of California-Riverside 








U of California-Santa Cruz 








U of Central Florida 


SUS 






U of Cincinnati 


SP Peer 






U of Colorado-Denver &Hlth Sci Ct 








U of Connecticut 


Big East 






U of Delaware 








U of Georgia 








U of Hawaii-Manoa 








U of Houston 








U of Idaho 








U of Illinois-Chicago 


SP Peer 






U of Kentucky 








U of Louisville 


Big East 






U of Maine 








U of Maryland-Baltimore County 








U of Massachusetts-Amherst 








U of Memphis 








U of Mississippi 








U of Nevada-Reno 
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Public 


Private 


U of New Hampshire 


Big East 






U of New Mexico- 








U of North Florida 


SUS 






U of North Texas 








U of Oklahoma Norman 








U of Rhode Island 








U of South Carolina-Columbia 








U of Southern Mississippi 








U of South Florida 


SUS, 

Big East 






U of Toledo 








U of Utah 








U of Vermont 








U of West Florida 


SUS 






U of Wisconsin-Milwaukee 








U of Wyoming 








Utah State U 








Virginia Polytech 








Virginia Commonwealth U 








Washington State U 








Wayne State U 








West Virginia U 


Big East 






Western Michigan U 
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PctCited 

PctWorldPapers 

Impact 5-yr 

ImpactOfCited 

Citations 

Papers 

TotEnrIOS 

TotDegOS 

CoreRev 

GrantAmount 

GrantPct 

Assets 

instExpse 

RschExpse 

FacBioHealth 

FacOthPhysical 

FacSocBeh 

FacEducation 

FacEngineering 

FacBusiness 

FacMultidiscipl 

FacLetrsNOth 

FACFineArts 

FACOther 

TotRschBio&HIth 

T otRschOthrPhysi cal 

TotRschSocBeh 

TotRschEngCompSci 

TotRschOther 

FedRschBio&HIth 

FedRschOthrPhysical 

FedRschSocBeh 

FedRschEngCompSci 

FedRschOther 

TARUAcadOS 

TARUAcad04 

TARUFacAwrdsOS 

TARUFacAwrds04 

TARUFedRsch04 

TARUFedRsch03 

TARUTotRsch04 

TARUTotRschOS 

TARUDocsOS 

TARUDocs04 

TARUPDOC03 

TARUPD0CO2 

Valid N (listwise) 



Table 6 

Descriptive Statistics for Variables 



N 








Mean 


Std. 


Variance 


Skewness 


Statistic 


Statistic 


Statistic 


Statistic 


Statistic 


Statistic 


Statistic 


Statistic 


Std. Error 


140 


.24 


.54 


.78 


.6846 


.05496 


.003 


-.376 


.205 


140 


.01 


.00 


.01 


.0021 


.00173 


.000 


1.616 


.205 


140 


10.34 


2.53 


12.87 


6.5971 


2.35588 


5.550 


.456 


.205 


140 


11.83 


4.67 


16.49 


9.4396 


2.70831 


7.335 


.400 


.205 


140 


551880.0 


1290.00 


553170.0 


64108.54 


72389.90 


5E+009 


2.940 


.205 


140 


42632.00 


342.00 


42974.00 


8188.450 


6791.102 


5E+007 


1.616 


.205 


145 


49616 


1996 


51612 


23404.10 


11232.46 


1E+008 


.445 


.201 


146 


12726 


459 


13185 


5278.86 


2693.527 


7255085 


.782 


.201 


146 


6E+009 


57552621 


6E+009 


9E+008 


9E+008 


8E+017 


2.928 


.201 


146 


******* 


1151052 


******* 


******* 


******* 


5E+016 


1.494 


.201 


146 


.550 


.020 


.570 


.25801 


.110038 


.012 


.088 


.201 


94 


2E+009 


33201086 


2E+009 


3E+008 


3E+008 


1E+017 


3.078 


.249 


140 


******* 


******* 


******* 


******* 


******* 


5E+016 


1.979 


.205 


140 


******* 


******** 


******* 


******* 


******* 


3E+015 


2.284 


.205 


146 


******** 


******** 


******** 


******** 


******** 


52827.45 


1.126 


.201 


146 


******** 


******** 


******** 


******** 


******** 


4127.421 


1.251 


.201 


146 


******** 


******** 


******** 


******** 


******** 


123965.0 


.936 


.201 


146 


******** 


******** 


******** 


******** 


******** 


14949.76 


.724 


.201 


146 


******** 


******** 


******** 


******** 


******** 


44924.05 
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